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The Standard Err^r of Equlpercentile Equating* 

V It is frequently desired to use scores on two or more forms of 

the same test interchangeably. If the test forms differ in difficulty o 

% f ♦ 

in other ways, some transformation .of the raw test scores must be made 
to adjust for these differences, transformations that (attempt to) make 
scores on different forms interchangeable are called equatings. In 
equipercentile equating, these transformations are determined by the 
requirement that for some specified population, the 'cumulative frequency 
distribution of the transformed scores shall (theoretically) be the same 
regardless of the te.st firm administer ed. 

In practice, some empirical procedure is used to implement this 
theoreti f a* / definition for an actual sample of examinees. Sampling 
fluctuation in the resulting empirical equating is the subject of 
concern here. 

Wnsider the following empirical large-sample equipercentile 
equating procedure: 

1. Administer tests X and Y to N examinees. Score both 
tests. 

2. Given any fixed x q , find the score y* that has the same 
sample cumulative frequency. 

\ 

3. Assert that the scores (x^y 1 ) are asymptotically equated. 
(This procedure is slightly biased since Nq observations lie below 
X q and only Nq - 1 observations lie below y* . We ignore this, * 
since it will not affect the asymptotic variance.) 



*This work was supported in part by contract N00014-80-C-0402 , ' 
project designation NR150-453 between the Office of Naval Research and 
Educational Testing Service. Reproduction in whole or in part is 
permitted for any purpose of the United States Government. 



We must now find the asymptotic sampliug variance of this y ! 

for fixed x . We will consider <first a (unrealistic) case where 
o 

the test score is a continuous variable, then a more usual case where 

the test score is nonnegative integer, 

/ 

ljf tests X and Y are given to the same examinees, there 
is l$cely to be a practice effect or a fatigue effect on the second 
test administered. To avoid- this, it is common to give tests X and 
Y to different random samples from the same population of examinees. 
We consider this case first. 

1. Continuous Case, Two Groups . 

Let F(£) and G(y) denote the cumulaftive*f reqilency distribution 
of score x and score y in the population. We administer test X 
to a Sample of examinees and find that in the sample a proportion 

q of these fall below the chosen fixed value x q . Having administered 
test Y to a sample of examinees from the same population, we 

denote the q -th order statistic in this sample by y % atid assert 
that y f is equivalent (equaled) to x q . We wish to fin<i the 
asymptotic sampling variance of y f (it is always to be understood 
that x q is fixed) Xf 

For fixed q , y f is a^rmptotically normally distributed with 
mean y t i determined by the relation 

X |q 



J 
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" q 



(l) 



r • 



and variance 



V )q S P^2 (8(y y'|q ))2 ' 



(2) 



where p = 1 - q and g(y) is the probability density at y 
(Kendall & Stuart, 1969, Sections 14.11-14.12) . When q is random, 
a well-known identity gives 



' Var y'-= Var( Uyt|q ) + | > • 



(3) 



/ 



From (1) 



d_ _ 1 

dq p y1q " 8^ y »| q ) 



By the delta method, we find 



Var(M ytjq ) = PQ/N lg ; 



(4) 



where Q is defined^hy- 




(5) 



G(y o ) = Q . 



(6) 
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To evaluate ^ G y f |q we rewrite^?), expand in series, and neglect 



higher order terms: 



So 2 » S 23 

y q 2 * 8 " 8 o x2 



= f -EL (i . 2^—^+ ... ) 

N 2 g D < - 8 o 

» S - -ij (P - Var p -' P 2 ) * (7) 

N 2 8 o K 2 8 6 N 2 8 o 



where g = 8(P y i| q ) • Substituting (4-) and (7) into (3) we have 
finally 



Var y 2 ( NT + Hi ) . (8) 
g 1 z 



i 

2. Discrete Case, Two Groups 



Consider next the case where scores x and y are nonnegative 

integers. Fotf convenience, we will always pick x q to be an integer 

plus 0.5. L6t F(x) and G(y) be distribution functions continuous 

to the right of each integer and let y Q be the integer defined by the 

relation C(y - 1) < Q 5 F(x ) < G(y ). It will ordinarily be 
o o — o 
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asymptotically infinitely unlikely that G(y Q ) - Q ; f or simplicity, 
we will assume hereafter that G(y Q ) # > Q . 

If we now define y f as in the preceding section, we are sure 
that integers^s^ 1 ^and y Q will be equal for sufficiently large N . 
This means to the usual order of approximation that y f will have an 
asymptotic variance of zero. Let us use linear interpolation to 
define a value y M as follows:^ 



y" -.y* 



- o.5+^_g. 

g 



(9) 




where G is the observed propor ti^n^of Y scores below the integer 

y* and g is the observed proportion of scores at y 1 . We will in 

the discrete case assert that y" is equated to^j^j< It is the ^symptotic 

variance of y lf that is now required. 



As noted above, the variance of y* is asymptotically zero. The 
proportions q , g , and G are asymptotically n^pnally distributed 



with known variances and covariaiices 



< 0 q = ?Q/H 1 



o - = 0 

qg 




o )/N 2 



2 S 
a. 



G~(l G )/N, 



- - g G /N 

gG~ 



Now 



dy " = ^3. _ dG_ 
g g 



q - G 
g 

1 



/ 



\ 



By the delta method, we obtain finally 



Var y" = [Var q + Var G + ( ^ — ) 2 Var 

8 o 8 ° 



+ 2 Cov (£,G )] 



*o 

-I2 I + -i- ?[G --Q 2 + ,i2^] 

N l*o N 2 g o ■ 8 ° 



^ r Pg + Pa 1 . (C o ,Q)(Q-G) 

r 2 N. N_ ~ N_ (G - G~) J 

g o 1 2 2 

where G~ = G - g . Note that the last fraction in (10) approaches 
o o 

zero as g 0 . Thus as g becomes small, (10) approaches (8), th 

o o 



variance for the continuous case. 



4 

3. Discrete Cas'fe, One Group 



If there were no practice or fatigue effect, it would be more 

« 

efficient to administer^both tests to the same students. In order to 

see how much difference this would make, we derive the sampling • 
variante of (9) for this case. 

Let a , b , c , d , k , m be sample frequdncies as 

defined by the accompanying x < x q x > x - 

diagram and^let a , 3 , y > y Q b a 

Y » 5 , k , and y be y ■ y Q k • m , 

the corresponding population y < y Q y - d ^c 
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proportions. In the present notation q 

A 

and g = k + m , so (9) becomes 



= h + k + -d , 6" =' c + A , 



y' - 0.5 + 



« b + k - c 



k + m 



(11) 



The 5 sample frequencies are again asymptotically multivariate 

normal with the familiar variances' and covariances: a, = g (1 - g)/N 

b 



= -Sy/N , and so forth. 



As before, 




jyii _ db + dc , m - b + c 



k + m 



(k + m) 



,, k + b - c , , 
(k + m) Z „ 



-Using the delta method.^we finally obtain after some -algebra 



1 



Var y" = ^- [ yK + (0 / Y )g 7 +(B - y) 2 ] 



where g = y + k 
o 



(12) 



If x and y are independently distributed, (12) becomes the 
same as (10) with, N, ■ N« ■ N . 
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4. Continuous Case, One Group 

' t . ' , > - 

When x and y are x < x x > x 

o t o 

continuous, we de&l with sample y >"y f b x a 

frequencies a , b , c , d , y < y- „ - d c 

as defined in the accompanying diagram/ The corresponding population 
proportions are denoted by a , 6 , y > 5 ; for example 6 - $(x o ,y f ) 
where $(x,y) 'is the cumulative distribution function jof x and y * 
Since y f is to be the q -th ortler statistic, as in section 1, 

where q^E b + d , it follows- that given x , y 1 must always be 

*. " o « 

ctosen so that b = c . Fbr given x , then, the frequency disttibu- 
^ o* 

tion of y f is proportional to the probability of 'finding one p'erson 

at y f , c persons with x < x q and y > ,y f , and c persons with 

x > x q -and < y f . Writing MS H - 1 , this probability is proportional 

to ft g(y f ) times the*sum, over all possible values* of c , of the multi- 

normal probability 



Mi B B (a ± 3) - ^ , ^ (13) 



c! c! (M - 2c) ! 



Using Stirling's approximation to the factorials, thef distribution 
•of y f for even is 



13 I. 
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Taking logs under" the summation sign, we have that the asymptotic 
distribution of y* is proportional to 



M/2, 

Z .exp(log A) 
c-o 



wher^ j 



log A = log g(y') - -| log(l - 2$ - log 2tt - log M - log C 

+ M[p log^jr + (1 - 2C.) log(l - 6 - y) - 2C log C 

* '- (1 - 2C) log(l - 2C)] 

«> 

where C 5 c/N . 

, Expand A iti powers of y = y f - y^ and c 5 C - y q • Using 
a zero subscript to denote quantities evaluated at y -'(note that 
6 0 5 Y Q > but that dB/dy Q J dy/dy Q ) and dropping terms of lower order 
tl^an M , we find 

) 



log A = - 2Yq(1 M . [ \ {(2 Y ; - g o ) 2 + g 2(l - 2 Yo ) y 2 



~ 2(2 Y » - g Q ) yc + 2c 2 ] (15) 



where v* = dv. /dy . 
f o f b 'o 
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As in Feller (1950, Section VII/2), we/see 'from (15) that c and 
y' are asymptotically bivariate normal/ Writing A = 1 - 2y q , and h = 
2y q - g Q , we see f rom j(15J that 



2 2 




2 2 
h H-g^A 



to2(l-p 2 )-l2i , 



- h 



d - P >o y .o c ' V 



(16) 



(17) 



(18) 



where P^is the correlation between ^y' and c . Squaring the 
last equation land multiplying by the other two, we find that 



2 . 



h 2 + g 



1 - 



(19) 



Thus finally,- from (16) and (19), 



(200 



Note that as the correlation between x and y approaches 
1.0, the proportion Y Q approaches zero and a/ f in (20) vanishes* 
We may also see that in (12) when k + p E g Q (becomes small so tfiat 
0 Var y" for the discrete case with one group approaches (20). 

V 
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When the correlation between x and y is zero, (20) is' the same as 



(8) with N - N 2 . 



■5 . Numerical Results 



Formulas (8) and (20) for the continuous case are simpler than 
formulas (10) and (12) for the discrete case. We will give 
-first some numerical results from formulas- (8) and (20). 

A, Monte Carlo study was carried out by drawing N = 1000 pairs of 
pseudorandom standardized normal bivariate deviates ta (x,y)' ^from a popula- 

- tion with a correlation of p - .90' (this is a typical correlation between 

xy 

parallel test forms). From this sample of 1000 cases the equated Value 
of y' was found separately for x q = 0, 0.5,; 1.0, 1.5, 2.0, 2.5 . 
The f oregoing^was repeated 1000 times with independently drawn blvariate 
samples. For each given*' x , the empirical standard deviation s was 
computed. 

its , * 

The resulting standard errors (not variances) are presented 

in the fourth column of TabLe 1. The corresponding theoretical values 
from (20) are shown in the third "column. There i£ excellent agreement 
between theoretical and Monte Carlo results. 

The second column of Table 1 shows the standard error, according- 
to (8), when teats' X and Y are administered to different examinees 
rather than to the same examinees. We see that this does not entail 
as serious a loss of equating accuracy as might "have been feared. In 
view of the likelihood of practice and fatigue effects, it seems that the 
methods of Sections 1 and 2 should be used -whenever possible, rather ' 

than the methods of Sections 3 and 4. 

r ** i 

■■ • ■ \ 
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Table 1 

Standard Errors of Equip ercen tile Equating for 
Normally Distribt^ed Variables 



Standard Error 



Eq. (8) Jor (20), p' «0] 

A/ 



p - .90 
! xy 



0 

.5 
1.0 
1.5 

2>0 
2.5 



N, 

; l 

• P56 
.059 
.068 
.086 
^124 
.200 



1000 



Eq f (20) Monte Carlo 



.030 
.032 
.038 
.052 
.080 
.138 



.029 
.032 
.037 
.053 
.07$ 
.137 



-13- 

j 

For illustrative purposes, Table 2 shows the standard erroxs of 
an equating of a 5G-item M (Metropolitan Achievement Test) Word 
Analysis test to a 40-item C (Comprehensive Test of Basic Skills) 
Reading Vocabulary test. The data were drawn from the Anchor Test 
Study (Loret; Seder, Bianchini, and Vale,' 1974) in which bot* 
tests were administered to a group of 1406 sixth-grade students. The 
resulting bivariate distribution of number-right scores was smoothed 
by a method described by Lord (1980, Section 17.4). The correlation between 
'M and C was .88. The tabled values were computed from" (12), ignoring 
the smoothing. 

The standard deviation of number-right M scores for this group of 
sixth graders is 11.5. The standard error of measurement for M scores 
is 2.7. The standard error of equating is much smaller than the 
Standard error of measurement, ^* ''"<'.'• 

Table 3 provides an empirical comparison between equipereentj^fe & 
equating and conventional linear equating. . In this case,\Form VSAA of ' 
the 90-item SAT Verbal test had been administered to 2665 students, along 
with an 'anchor test 1 of 40 verbal items. At a later time, a new, 85 T 
item Verbal form, XSA2, was similarly administered along with the sajp6 - . \ 

anchor test to a new group of 2686 students. As part of normal goring and 
reporting, Form XSA2 raw ('formula') scores were equated standard 
linear method due to L. R Tucker (Angoff , 1971, Equating Design itf.A*) 
to the scaled scores on Form VSA4. This equating is shown along with its { 
standard error as determined by the computer program AUTEST (Lord, 1975), 
in the first three columns of Table 3. 



l0 



1 



* k Table 2 

Standard Error of .Equipercentile Equating, 

MAT to CTBS . 





Cumulative . 


Equated 


' Standard 


c 


Frequency 


M 


Error of 


Scores 


Distribution 


Scores 


Equating 


37.5 


.98 


49.3 


.17 


32.5 


.89 


45.8 


.20 


27.5 


.76 


42.1 


4 22 


22.5 


.61 


37.6 


.26 


17.5 


.44 


31.9 


.32 


12.5 


.26 


23.1 


.44 ' 


7.5 


.08 


13.8 


.36 


2.5 . 


,01 


8.8 


.44 



♦ 



Number-Right Scores K 
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Table ' / 
Comparison of Linear and Equiirercentile Equating for the Verbal Score on 
Form XSA2, Collegfe Board Scholastic Aptitude Test / 



Linear (Tucker) Model Equipercentile/Method 



XSA2 


Equivalent 




Equivalent 




'f ormula 


scaled 


Standard 


scaled 


^Standard 


score 


score 


error 


score 


j error 


78.1 


738 


4.07 


774 , 


/ 13.47 


70.6 


685 


3.46' 


722 ' / 


15.85 


64.75 ■ 


644 


3.00 


652 / 


10.32 


58.9 


602 


2.56 


' 602 / 


4.97 


52.-9 , 


559 


2.15 


558 / 


4. 12 


47.25 


519 


1.82 


514 / 
466 / 


3.47 


■ 40,1 


469 


1.54 


3.44 


32.4 


414 


1.51" 


417 / 


2.93 


25.75 


367 


1.72 


364 / 


3.37 


16.1 


298 


2.28 


314 / 


4.07 


7.6 


238 


2.90 


242 / 


5.70 


-3.75 


157 


3.80 


195 / 


7.85 



■ J . . , \ 
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The equipercentile equating of XSA2 to VSA4 was effected by equating 
each test to the anchor test independently and then using the rule that 
scores equated to the same anchor-test score are equated to each other. 
Thus the equipercentile equating of XSA2 to VSA4 requires two independent 
equatings of the type treated in Section 3 of this paper. The sampling 
variances (12). of the two equatings are additive, since they are computed 
from two different samples of students. The resulting equipercentile 
equating and standard error are shown in the last two columns of the 
table. 

To put these standard errors into perspective, note that the 
standard deviation of scaled scores in a group of students is typically 
about 100. The standard error of measurement of a scaled score is about 
30 to 33. The standard errors o*f equating are mostly small by comparison. 

The standard errors of the equipercentile equating are double those of the 

\ 

linear equating in the middle of the score range, comparatively larger at 
i 

the extremes. 

Equipercentile equating can be improved by smoothing the empirical 
frequency distribution of scores before equating. This reduces the 
sampling errors but may introduce, small biases that do not disappear even 
in very large samples* ) The sampling etfox of a smoothed equating could 
perhaps be determined for a specified smoothing method, but the mathematics 
would be burdensome. 

4 
f 
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